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Abstract 

In two experiments, we investigated whether advantages of studying graphic organizers over outlines for 
learning text structure and transfer from a chapter-length text are restricted to study sequences that employ 
delayed, rather than immediate, review. Undergraduates studied one of three sets of study materials, 
reviewed them immediately or two days later, and then were tested. Results indicated that although graphic 
organizers facilitated learning text structure across both review occasions, they facilitated transfer of text 
knowledge only when review was delayed. We also found evidence that students who delayed their review 
of graphic organizers used more effective study strategies. These findings are explained within the 
framework of the deactivation hypothesis (Krug, Davis, & Glover, 1990). 



Graphic organizers (GOs) are spatial displays of text information that can be provided to students 
as study aids (i.e., adjunct displays) that accompany text. GOs, like the one in Figure 1, communicate both 
vertical, hierarchical concept relations (e.g., neuroses is a type of abnormal behavior, is characterized by 
severe distress and lack of control over one's life, etc.), and also horizontal, coordinate concept relations 
(e.g., persons with neuroses are aware of their problems, whereas persons with psychoses and personality 
disorders are not). Although the outline displayed in Figure 2 contains the same information as the GO in 
Figure 1, recent evidence suggests that it is not as effective as the GO in communicating concept relations 
because it uses a linear, rather than spatial, format (Robinson & Kiewra, 1995; Robinson & Schraw, 1994). 
Winn ( 1990) has suggested that students may extract more information from a quick glance at a spatial 
display than they can from a longer viewing of a linear display. Unfortunately, most textbooks still contain 
outlines at the beginning of chapters, most students still take notes in an outline format, and most teachers 
write outlines of their lectures on the chalkboard. The present study was designed to investigate how GOs 
may be more effective than outlines as adjunct displays in environments similar to actual classroom 
learning; that is, where students read chapters of text and then review before being tested. 

Graphic Organizers are better than Outlines 

Several recent studies have examined the instructional advantage of providing GOs, rather than 
outlines, to students as adjunct displays. Robinson and Schraw (1994) had students read a short (200 words) 
text and then study either the same text, an outline, or a GO. Students who studied the GO learned more 
concept relations than those who studied only the text when tested immediately, even when the time to 
study was reduced to only one minute. When testing was delayed for 10 minutes, however, there were no 
advantages of studying GOs. Robinson and Schraw hypothesized that computing concept relations when 
studying outlines and texts requires more effort than when studying GOs. They concluded that the GO's 
advantage in computational efficiency (Larkin & Simon, 1987) may actually be detrimental to long-term 
(delayed) learning because too little effort is used during encoding. 

Computational efficiency suggests that the facilitative advantage of GOs occurs during encoding, 
when concept relations are computed and then stored in memory as new propositions. Others have argued, 
however, that the facilitative advantage of spatial displays occurs during retrieval, when an additional 
spatial memory representation becomes available along with a verbal representation (e.g., Kulhavy, Lee, & 
Caterino, 1985). This possibility that the same information contained in different adjunct displays may be 
encoded in memory in different formats was examined in a study by Robinson, Katayama, and Fan ( 1996) 
who employed dual task methodology. Students studied one of four types of comprehension displays (a 
text, an outline, a GO, or a concept map), viewed either a spatial (set of dots) or verbal (set of numbers) 
memory display, and then were tested on comprehension of the first, and memory of the second display. 
When students viewed GOs or concept maps, recognition of the spatial display was inhibited, whereas 
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viewing texts or outlines did not interfere with memory of the spatial display. Robinson et al. concluded that 
because GOs and concept maps are more spatial than outlines and texts, they are encoded in memory in a 
more spatial format than are the more linear displays. Additional support for this claim has recently been 
reported by Robinson, Robinson, and Katayama (submitted). 

If one assumes that students search their mental representations when attempting to answer 
questions about information presented in the text, then the more efficiently organized the mental 
representation, the more likely the student will be able to retrieve the necessary information. Although no 
studies have directly examined exactly how students mentally retrieve information (which would be very 
difficult to do), there have been recent studies where students retrieve information from external (on paper) 
representations to answer comprehension questions. Robinson and Skinner (1996) found that when students 
searched GOs, they found information needed to answers questions faster than when they searched outlines 
or texts. These results provide additional evidence that GOs are computationally more efficient than 
outlines or texts, facilitating both local (searching for concept facts) and global (searching for concept 
relations and patterns among concept attributes) search tasks. Thus, one possible explanation for why GOs 
facilitate learning of concept relations is that they are mentally stored in an efficient, spatial format that can 
be easily searched for information, sort of like a library that is neatly organized as opposed to one where 
books are stacked in piles. 

Although the practice of including GOs in textbooks and study guides has been increasing, most of 
the research studies that have investigated their instructional potential have failed to examine them in 
authentic learning environments (see Robinson, in press, for a review). "Typically studies have used short, 
poorly organized text, single GOs, and immediate tests measuring only factual knowledge" (Robinson & 
Kiewra, 1995, p. 455). Robinson and Kiewra (1995) attempted to address these weaknesses by investigating 
the relative effectiveness of multiple adjunct displays (outlines and GOs) in facilitating students' learning of 
concept facts, relations, and transfer when studied in conjunction with a chapter-length, well-organized text. 
To encourage students to become more involved with the content of the text, instructional directives were 
inserted in the text that directed the students to view the adjunct displays. 

In Experiment 1, students studied either the text alone, the text plus outlines, or the text plus GOs 
for one hour and then were tested either immediately or two days later. Two noteworthy findings were 
observed. First, students who studied GOs learned more coordinate concept relations than the outline and 
text only groups, and expressed more of them in a contrastive manner than the outline group. This finding 
demonstrated that GOs also facilitate learning of coordinate concept relations for longer text. Second, on the 
application test which measured transfer, for students who studied only the text, there was a decrease in 
performance from immediate to delayed testing, whereas the GO group experienced no decrease. This 
Finding supported the delay hypothesis (Andre, 1990), which suggests that adjunct aids' facilitative effects 
are more resistant to a delay. Moreover, because a GO advantage was observed across both immediate and 
delayed testing conditions, these results suggested that for chapter-length text and multiple GOs, 
computational efficiency of the GOs is not detrimental to long-term learning, as suggested by Robinson and 
Schraw (1994). 

Although some new advantages to studying GOs were found, GOs were not shown to be more 
effective for either learning hierarchical relations (text structure) or applying knowledge to solve problems 
(transfer). Robinson and Kiewra (1995) speculated that students in the GO group may not have had enough 
time to take full advantage of the GOs. To test this hypothesis, in Experiment 2, students were given one 
hour to study and then two days later had an additional 15 minutes to review before being tested. It was 
hypothesized that this study sequence would avoid problems associated with fatigue (i.e., subject tolerance) 
and be more similar to what students actually do. This time the GO group outperformed both the text only 
and outline groups on the hierarchical relations and application tests. Robinson and Kiewra concluded that 
the appearance of these differences in Experiment 2 could be attributed to students receiving adequate time 
to study their materials, based on the fact that only 8% of the students in the GO and outline groups 
indicated that they did not have adequate time to study the displays as compared to 40% in Experiment 1. 
They failed to consider, however, the possibility that this appearance of differences may have simply been 
due to delaying review, rather than increasing study time. It is this issue that motivated the present study. 
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Delayed Review is better than Immediate Review 

The study sequence employed by Robinson and Kiewra (1995) in Experiment 2 involved 
interspersing two exposures to the same material with a delay. In effect, they were using a distributed 
practice sequence rather than the massed practice sequence used in Experiment 1 where the two exposures 
were not separated by a delay. Distributed practice has long been linked to the spacing effect - a learning 
phenomenon where, given the same amount of time, persons who intersperse study sessions with delays 
learn better than those whose study sessions are massed together with no delays (see Dempster & Ferris, 
1990, for a more detailed account). 

Although many factors may be involved in producing the spacing effect (Anderson, 1990), two 
explanations have been identified to date. The first explains how forgetting can be beneficial for long-term 
retention. The "deactivation hypothesis" (Krug, Davis, & Glover, 1990) states that when information is first 
encountered, full processing occurs, the information becomes "active" in memory, and then soon begins to 
"deactivate". If the same information is encountered a second time while it is still active in memory, only 
partial processing or activation will occur. If, however, the information is encountered a second time after it 
has deactivated, full processing will again occur, strengthening the memory structure more than if only 
partial processing had occurred. Stronger memory structures are more likely to be retrieved than weak ones 
(Anderson, 1990). 

A second possible explanation for the spacing effect is based on the idea that additional information 
in the learning environment (e.g., surroundings, lighting, temperature, etc.) is encoded and associated in 
memory with the target information (i.e., what is supposed to be learned). Because contextual elements are 
associated with the target information in memory, they may serve as retrieval cues. If the test context 
overlaps with the study context, additional retrieval cues will be available and test performance will 
increase. Godden and Baddeley (1975) demonstrated this fact in a rather unique way. Divers learned a list 
of words either on land or under water and then attempted to recall the words either on land or under water. 
Those whose test environment was the same as their learning environment had the best recall performance. 
When information is studied on different occasions, each occasion involves encoding different contextual 
cues. When study occasions are distributed over time, the study contexts will differ more from each other 
than if the study occasions are massed together. The more two study contexts differ, the more likely one of 
the contexts will overlap with the test context. Different contexts mean more contextual cues and increase 
the probability that a particular cue may be present at the time of retrieval (Anderson, 1990). 

Thirty-six years ago, Underwood (1961) concluded that the spacing effect is unimpressive because 
it is restricted to very specific conditions and the effect itself is small. In studies conducted since then, 
however, the spacing effect has been found to be a robust and powerful phenomenon that generalizes to 
textbook material (Anderson, 1990). Unfortunately, evidence suggests that educators may still believe the 
earlier conclusions of Underwood. Zechmeister and Shaughnessy (1980) found that many people believe 
that massed review is better than spaced review. Dempster and Ferris (1990) even noted that one influential 
educator (Hunter, 1983) has advised against spaced review. Dempster and Ferris concluded that the full 
potential of the spacing effect has not been realized mainly because reviews are not always popular among 
teachers. In the present study, we wanted to see if the previously observed advantages of studying graphic 
organizers in addition to chapter-length text are restricted to study sequences that employ delayed, rather 
than immediate, review. 



Experiment 1 

Experiment 1 was to designed to investigate whether the facilitative advantages of GOs over 
outlines and text-only for learning hierarchical concept relations and transfer observed by Robinson and 
Kiewra (1995, second experiment) may have been due to delayed review rather than increased study time. 
Our study was very similar to their experiment, except that we directly compared immediate and delayed 
review conditions. Most importantly, we gave students in both the immediate and delayed review groups 
the same amount of total study time to examine the effects of review occasion absent any increases in study 
time. Finally, because we were mainly interested in finding differences between the GO and outline groups, 
students were administered only the hierarchical relations and application tests. Robinson and Kiewra found 
no differences between outlines and GOs on the facts tests, and the same advantage for the GO group over 
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the outline group on the coordinate concept relations measure for both immediate and delayed review 
conditions. 



Method 



Participants and Design 

Eighty-nine students enrolled in an undergraduate educational psychology course at a state 
university in the South were assigned randomly to one cell of a 3 by 2 design (cell sizes are displayed in 
Table 1). The first factor was study materials (text-only, text-plus-outlines, or text-plus-GOs), and the 
second factor was review occasion (immediate or 2-day delay). Testing took place in groups of about 30 
students, mixed with respect to study materials conditions, in a typical university classroom. 

Materials 



The materials were identical to those used by Robinson and Kiewra (1995). 

Text . A 6500-word text (12 pages, single-spaced) on abnormal behavior was used. Most of the 
material was taken from an undergraduate psychology textbook (Davidoff, 1976), with a section on 
psychopathic behavior taken from a similar textbook (Rubin & McNeil, 1981). Two versions of the text 
were used, one for the text-only group and one for the two adjunct displays groups. The latter version 
included seven additional sentences, each placed at the end of sections, that directed the reader to view one 
of the seven displays. 

Adjunct displays . Seven adjunct displays, in both GO (six matrices and one tree diagram) and 
outline format, were used. Examples of both types of displays are shown in Figures 1 and 2. 

Tests . Two tests were used. The hierarchical relations test contained 21 cued-recall items requiring 
knowledge of text structure. It was printed over four pages, with items arranged so they would not provide 
clues to other items on the same page or on succeeding pages. The application test contained 15 matching 
items that required identification of disorders, given novel examples of symptoms. Names of all 16 
disorders mentioned in the text appeared in an alphabetical list, and students were instructed that some 
names may have been used more than once or not at all. Examples of each test item appear below. 

Hierarchical Relations 

List the four types of schizophrenia described in the text. 

Application 

Karen spent most of her day hanging out at the local beauty parlor. The other women were 
becoming tired of hearing her outrageous stories. For example, Karen once told them that she, not 
Queen Elizabeth, was the rightful heiress to the British throne and that she had been denied her 
privileges by the CIA. At another time she insisted that it was she, and not Jonas Salk, that had 
developed the polio vaccine. 

Procedure 



Sessions were conducted in a single 80-minute period for the immediate groups and a 50-minute 
period combined with a 30-minute period 2 days later for the delayed groups. First, all students were 
randomly assigned to either the immediate or the delayed conditions. Students in the immediate condition 
were excused and told to return two days later. Students in the delayed condition were given envelopes 
containing study materials and wrote their names on labels affixed to the envelopes. The envelopes were 
arranged in a systematic order to ensure approximately equal numbers of participants in each study 
materials condition. Students were told they had 50 minutes to read and study their materials. After the 50 
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minutes had expired, students placed their materials back into the envelopes, were told not to discuss the 
experiment, and were excused. 

Two days later, the students in the immediate condition returned and were given envelopes 
containing the study materials. They were also told that they had 50 minutes to read and study the materials. 
After the 50 minutes had expired, students placed their materials back into the envelopes and remained 
seated at their desks. Students in the delayed condition entered the classroom and were seated. The 
experimenter read their names from the labeled envelopes and gave them their study materials. Next, all 
students were given 10 minutes to review their materials. After the 10 minutes had expired, all study 
materials were collected from the students. The experimenter distributed a series of test sheets (hierarchical 
relations) which students completed one at a time, placing them in their envelopes, so that they could not 
return to complete an earlier test based upon knowledge gained from subsequent test sheets. This procedure 
lasted about 10 minutes. Students then completed the application test in about 10 minutes and placed it in 
their envelopes. Students in the delayed condition were told to remove their labels so that performances 
could not be associated with names. 

Scoring 



All tests were scored by (the second author) without knowledge of group affiliation (tests were 
previously coded) in accordance with predetermined keys, with a ipaximum score equal to the number of 
items. 

Results 

Table 1 presents means and standard deviations for scores on the two tests. All statistical tests were 
conducted at the alpha = .05 level of significance. For each test, a 3 (study materials) by 2 (review occasion) 
factorial analysis of variance (ANOVA) was conducted. 

Hierarchical Relations 



The main effects of study materials and review were not significant, F(2, 82) = 1.85 and F( 1, 82) = 
.29, respectively. The interaction effect of study materials by review, however, was significant, F(2, 82) = 
3.41, MSE = 15.21, p= .038 (see Figure 3). Once again, no tests of simple effects comparing the three study 
materials groups within each review occasion condition were significant. However, simple effects tests 
comparing the review occasion groups within each study materials condition indicated that for both the text- 
only [F(l, 82) = 1 . 71 , 2 = . 19] and the outline conditions [F(l, 82) = .00, 2 = .96] the effect of review was 
not significant, whereas for the graphic organizer condition the delayed review group scored higher than the 
immediate review group, [F(l, 82) = 5.48, £ = . 022 ]. 

Application 

The main effects of study materials and review were not significant, F(2, 82) = .24 and F( 1, 82) = 
.36, respectively. The interaction effect of study materials by review occasion, however, was significant, 
F(2, 82) = 3.93, MSE = 8. 10, p = 024 (see Figure 4). Tests of simple effects were then conducted to see if 
the three study materials groups differed within each review occasion condition. None of these simple 
effects were significant, so a second set of simple effects tests were conducted to see if the two review 
conditions differed within each study materials condition. For both the text-only [F( 1 , 82) = 2.33, p = . 13] 
and the outline conditions [F(l, 82) = .04, p= .85] the effect of review was not significant. However, for the 
graphic organizer condition the delayed review group scored higher than the immediate review group, [F(l, 
82) = 5.94, p = .017]. 



Discussion 

The present study investigated whether facilitative effects of delaying review for learning text 
structure and transfer depend on the type of study materials reviewed. Undergraduates studied either a 
chapter-length text, the text plus a set of GOs, or the text plus a set of outlines. Students were given 50 
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minutes to study their materials, and then either immediately after or two days later they studied for an 
additional ten minutes. Students were then tested over knowledge of text structure and transfer. Results 
indicated that for those who studied text plus GOs, those who delayed review performed better than those 
who immediately reviewed. For students who studied either text-only or text plus outlines, performances 
did not differ among the immediate and delayed review groups. These results are consistent with those 
obtained by Robinson and Kiewra ( 1995) in terms of differences between Experiments 1 and 2. But more 
importantly, because both review groups were given the same amount of study time, the results of the 
present study suggest that GOs' facilitative effects for learning text structure and transfer depend on 
delaying review, and not on additional study time as suggested by Robinson and Kiewra. 

Why does delaying review facilitate learning from GOs, but not outlines or text? One reason may 
have to do with the nature of the learning task. Mumford, Costanza, Baughman, Threlfall, and Fleishman 
(1994) had students perform spatial visualization and perceptual speed tasks. Students then practiced a 
complex skill by either massing or distributing their practice. Perceptual speed facilitated performance for 
those who massed their practice, whereas spatial visualization facilitated performance for those who 
distributed their practice. Mumford et al. concluded that distributed practice facilitates spatial visualization, 
whereas massed practice facilitates associational learning. 

Robinson, Katayama, and Robinson (in preparation) investigated the possibility that information 
contained in spatial displays, such as GOs and concept maps, may be represented in memory in a different 
format than that information contained in linear displays, such as outlines and texts. In one of the 
experiments, students viewed one of the four types of displays and then viewed a either a spatial 
configuration of dots or a verbal list of digits. They then answered a comprehension question about the 
display and attempted to identify either the dots or digits they had previously seen. Viewing dots inhibited 
comprehension performance for GOs and concept maps, whereas viewing digits inhibited comprehension 
performance for outlines or texts. Robinson et al. (submitted) concluded that retrieving information that is 
encoded as a result of viewing spatial displays involves the use of spatial working memory. Information 
encoded as a results of viewing linear displays, however, involves the use of verbal working memory. This 
evidence, when combined with the findings of Mumford et al. (1994), suggests that because studying GOs 
appears to be a more spatial, rather than associational task, distributed practice is beneficial. However, 
because studying outlines and texts is not a spatial task, and in fact may be a more associational task, there 
is no advantage for distributing practice. 

Based on the deactivation hypothesis (Krug et al., 1990), we offer the following explanation to 
account for the present findings. When students study only the text or outlines, they attempt to simply 
associate facts with concept names and then rehearse those associations. When review is immediate, they 
have just completed reading the text and the association strategy works well because memory for the 
associations is still active. The active memory of these associations is then used to answer questions about 
more complex associations like hierarchical concept relations. The point here is that because memory for 
the individual associations is still active, students can be moderately successful performing more complex 
retrieval tasks. However, when practice is distributed, the associations that were learned on the initial study 
session have deactivated and students must spend the short, 10-minute study period attempting to reactivate 
each of the several previously learned associations. Although the results of the present study did not 
indicate a statistically significant advantage for immediate review over delayed review, students who 
studied only the text had their best performances when study was massed. 

When students study GOs, they attempt to visualize the GO and learn the overall structure of the 
material. They do not concentrate on specific associations, but rather they focus on hierarchical and 
coordinate concept relations. When review is immediate, students’ memory for the text is still active and 
they are not able to perform the deep, elaborative processing required to learn the concept relations. This is 
because the development of knowledge structures requires that memory for individual associations (concept 
facts) must deactivate so more complex relations can be learned (Mumford et al., 1994). When review is 
delayed, however, students’ memory for associations has deactivated and they can focus on learning the 
overall structure of the material. 

Because most GO research has used short texts and tested only knowledge of facts immediately 
after study, the combined benefits of GOs and delayed review have not been previously shown. The main 
advantage to studying GOs may lie in the type of processing they encourage. When students are given text 
or a text plus outlines, they may attempt to associate concept names and facts rather than attempting to 
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understand the "big picture". When students are given GOs, however, they may attempt to learn the 
structure rather than the facts. The implications of the present study are important because no one has 
considered how GOs should be provided to students. Moore and Readance ( 1980) conducted a meta- 
analysis of studies investigating GOs and concluded that GOs are most effective when presented after the 
text has been read, rather than before. We recommend that GOs be presented perhaps a few days after the 
text has been read or at least until students' memory for the text has deactivated. With a delayed review, 
students would be able to focus on relations communicated by the GOs instead of trying to remember the 
surface structure of the text. 

Although the present study addressed a possible confounding variable (different study time) in the 
Robinson and Kiewra (1995) study, it, too, has a few limitations. First we could not be sure that students 
were spending equal amounts of time studying the GOs and outlines. Some students finished reading the 
text early and thus had more time to study the adjunct displays. These differences were controlled 
somewhat, but not completely, by random assignment. Second, providing students with materials may be a 
good idea but most would agree that an even better idea would be to have students construct GO notes 
themselves. Currently, we are developing a study to investigate the effects of having students learn to 
construct GOs on their own and then study them after a delay. Although this practice may not be more 
authentic than simply providing notes, we feel that is a potentially more effective learning activity. 
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Table 1 

Means (and Standard Deviations) for Percent Correct Scores by Group in Experiment 1. 

Study Materials 

Review Test Text-only Outline Graphic Organizer 



Immediate 


Hierarchical Relations 


26.52(16.38) 


31.43(21.14) 


19.67(13.90) 




Application 


48.07 (26.07) 


43.13(17.07) 


32.00(15.13) 




n 


14 


15 


15 


Delayed 


Hierarchical Relations 


17.33(13.14) 


31.10(19.19) 


35.57(24.52) 




Application 


37.13(15.87) 


44.47(17.07) 


48.87 (20.40) 




n 


14 


15 


15 


Table 2 










/ 

Means (and Standard Deviations) for Percent Correct Scores by Group in Experiment 2. 



Study Materials 



Review 


Test 


Text-only 


Outline 


Graphic Organizer 


Immediate 


Hierarchical Relations 


41.67(33.33) 


61.42(30.50) 


66.25 (22.83) 




Application 


70.50(21.14) 


74.74 (26.32) 


68.95(16.96) 




n 


20 


19 


19 


Delayed 


Hierarchical Relations 


44.83 (33.75) 


57.42 (30.00) 


70.83 (23.42) 




Application 


61.88(26.64) 


70.00 (20.58) 


85.00 (20.07) 




n 


16 


18 


18 




11 
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